Joint source channel coding for progressive image transmission

Recent wavelet-based image compression algorithms achieve best ever performances with fully embedded bit streams. However, those embedded bit streams are very sensitive to channel noise and protections from channel coding are necessary. Typical error correcting capability of channel codes varies according to different channel conditions. Thus, separate design leads to performance degradation relative to what could be achieved through joint design. In joint source-channel coding schemes, the choice of source coding parameters may vary over time and channel conditions. In this research, we proposed a general approach for the evaluation of such joint source-channel coding scheme. Instead of using the average peak signal to noise ratio (PSNR) or distortion as the performance metric, we represent the system performance by its average error-free source coding rate, which is further shown to be an equivalent metric in the optimization problems. The transmissions of embedded image bit streams over memory channels and binary symmetric channels (BSCs) are investigated in this dissertation. Mathematical models were obtained in closed-form by error sequence analysis (ESA). Not surprisingly, models for BSCs are just special cases for those of memory channels. It is also discovered that existing techniques for performance evaluation on memory channels are special cases of this new approach. We further extend the idea to the unequal error protection (UEP) of embedded images sources in BSCs. The optimization problems are completely defined and solved. Compared to the equal error protection (EEP) schemes, about 0.3 dB performance gain is achieved by UEP for typical BSCs. For some memory channel conditions, the performance improvements can be up to 3 dB. Transmission of embedded image bit streams in channels with feedback are also investigated based on the model for memory channels. Compared to the best possible performance achieved on feed forward transmission, feedback leads to about 1.7 dB performance improvement

Joint Topic-Semantic-aware Social Recommendation for Online Voting

Online voting is an emerging feature in social networks, in which users can express their attitudes toward various issues and show their unique interest. Online voting imposes new challenges on recommendation, because the propagation of votings heavily depends on the structure of social networks as well as the content of votings. In this paper, we investigate how to utilize these two factors in a comprehensive manner when doing voting recommendation. First, due to the fact that existing text mining methods such as topic model and semantic model cannot well process the content of votings that is typically short and ambiguous, we propose a novel Topic-Enhanced Word Embedding (TEWE) method to learn word and document representation by jointly considering their topics and semantics. Then we propose our Joint Topic-Semantic-aware social Matrix Factorization (JTS-MF) model for voting recommendation. JTS-MF model calculates similarity among users and votings by combining their TEWE representation and structural information of social networks, and preserves this topic-semantic-social similarity during matrix factorization. To evaluate the performance of TEWE representation and JTS-MF model, we conduct extensive experiments on real online voting dataset. The results prove the efficacy of our approach against several state-of-the-art baselines.Comment: The 26th ACM International Conference on Information and Knowledge Management (CIKM 2017

CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning

Nowadays, the research on Large Vision-Language Models (LVLMs) has been significantly promoted thanks to the success of Large Language Models (LLM). Nevertheless, these Vision-Language Models (VLMs) are suffering from the drawback of hallucination -- due to insufficient understanding of vision and language modalities, VLMs may generate incorrect perception information when doing downstream applications, for example, captioning a non-existent entity. To address the hallucination phenomenon, on the one hand, we introduce a Contrastive Instruction Evaluation Method (CIEM), which is an automatic pipeline that leverages an annotated image-text dataset coupled with an LLM to generate factual/contrastive question-answer pairs for the evaluation of the hallucination of VLMs. On the other hand, based on CIEM, we further propose a new instruction tuning method called CIT (the abbreviation of Contrastive Instruction Tuning) to alleviate the hallucination of VLMs by automatically producing high-quality factual/contrastive question-answer pairs and corresponding justifications for model tuning. Through extensive experiments on CIEM and CIT, we pinpoint the hallucination issues commonly present in existing VLMs, the disability of the current instruction-tuning dataset to handle the hallucination phenomenon and the superiority of CIT-tuned VLMs over both CIEM and public datasets

RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems

To address the sparsity and cold start problem of collaborative filtering, researchers usually make use of side information, such as social networks or item attributes, to improve recommendation performance. This paper considers the knowledge graph as the source of side information. To address the limitations of existing embedding-based and path-based methods for knowledge-graph-aware recommendation, we propose Ripple Network, an end-to-end framework that naturally incorporates the knowledge graph into recommender systems. Similar to actual ripples propagating on the surface of water, Ripple Network stimulates the propagation of user preferences over the set of knowledge entities by automatically and iteratively extending a user's potential interests along links in the knowledge graph. The multiple "ripples" activated by a user's historically clicked items are thus superposed to form the preference distribution of the user with respect to a candidate item, which could be used for predicting the final clicking probability. Through extensive experiments on real-world datasets, we demonstrate that Ripple Network achieves substantial gains in a variety of scenarios, including movie, book and news recommendation, over several state-of-the-art baselines.Comment: CIKM 201

Privacy-Preserving Face Recognition Using Random Frequency Components

The ubiquitous use of face recognition has sparked increasing privacy concerns, as unauthorized access to sensitive face images could compromise the information of individuals. This paper presents an in-depth study of the privacy protection of face images' visual information and against recovery. Drawing on the perceptual disparity between humans and models, we propose to conceal visual information by pruning human-perceivable low-frequency components. For impeding recovery, we first elucidate the seeming paradox between reducing model-exploitable information and retaining high recognition accuracy. Based on recent theoretical insights and our observation on model attention, we propose a solution to the dilemma, by advocating for the training and inference of recognition models on randomly selected frequency components. We distill our findings into a novel privacy-preserving face recognition method, PartialFace. Extensive experiments demonstrate that PartialFace effectively balances privacy protection goals and recognition accuracy. Code is available at: https://github.com/Tencent/TFace.Comment: ICCV 202

MARS: Exploiting Multi-Level Parallelism for DNN Workloads on Adaptive Multi-Accelerator Systems

Along with the fast evolution of deep neural networks, the hardware system is also developing rapidly. As a promising solution achieving high scalability and low manufacturing cost, multi-accelerator systems widely exist in data centers, cloud platforms, and SoCs. Thus, a challenging problem arises in multi-accelerator systems: selecting a proper combination of accelerators from available designs and searching for efficient DNN mapping strategies. To this end, we propose MARS, a novel mapping framework that can perform computation-aware accelerator selection, and apply communication-aware sharding strategies to maximize parallelism. Experimental results show that MARS can achieve 32.2% latency reduction on average for typical DNN workloads compared to the baseline, and 59.4% latency reduction on heterogeneous models compared to the corresponding state-of-the-art method.Comment: Accepted by 60th DA

Virtual carbon and water flows embodied in global fashion trade - a case study of denim products

The environmental impacts of the fashion industry have been aroused wide concerns. The globalization and fragmentation of the textile and fashion system have led to the uneven distribution of environmental consequences. As denim is the fabric of jeans that is representative of fashion, this study assessed virtual carbon and water flows embodied in the global denim-product trade, and footprints of denim production were quantified by life-cycle assessment and water footprint assessment. Results indicated that virtual carbon embodied in the global denim trade increased obviously from 14.8 Mt CO2e in 2001 to 16.0 Mt CO2e in 2018, and the virtual water consumption dropped from 5.6 billion m3 to 4.7 billion m3 from 2001 to 2018. The denim fabric production and cotton fibre production respectively contribute the most of the carbon emissions and water consumption. Polyester blended denim has 5% larger carbon footprint and 72% lower water footprint than cotton denim, and contributes to increasing embodied carbon emissions (from 4% in 2001 to 43% in 2018). Increasing the utilization of polyester blended denim would save water but face more pressures on carbon emission reduction. In the past two decades, virtual carbon and water flows embodied in the global denim trade are relocating, main jean consumers (i.e., the USA, EU-15, and Japan) withdraw the denim manufacturing supply chain and developing countries (i.e., China, India, and Pakistan) with higher carbon and water footprint undertake main global denim production, facing increasing climate-related risks and water crisis. The South-South cooperation helps share successful experiences, save production cost, and lessen resource consumption and environmental emissions. The production and consumption of denim should be shifted to circular and sustainable ways and new business models are required. The analysis framework can provide the basis for exploring environmental flows of product-level trade, and results can offer a basis for environmental policies and control strategies of the fashion industry, and as well as the sustainable production and consumption of garment